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Abstract 

An expression for the moment of partition function valid for any finite system size 
N and complex power n (Re(n) > 0) is obtained for a simple spin glass model termed 
the discrete random energy model (DREM). We investigate the behavior of the moment 
in the thermodynamic limit N oo using this expression, and find that a phase 
transition occurs at a certain real replica number when the temperature is sufficiently 
low, directly clarifying the scenario of replica symmetry breaking of DREM in the 
replica number space without using the replica trick. The validity of the expression is 
numerically confirmed. 
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§1. Introduction 



The replica method (RM) is one of the few analytic schemes available for the research of 
disordered systemsP In physics, this method has been well known since the 1970s and has 
been successfully applied to the analyses of spin-glass modelsP''^''^ although the essential 
idea behind the method can be dated back to the end of 1920s where it appeared as a the- 
orem for computing the average of logarithmsP''^''^ More recently, considerable attention 
has been paid to the similarity between statistical mechanics of disordered systems and the 
Bayesian method in problems related to information processing (IP)P The number and 
variety of applications of RM to problems the IP research are increasing rapidly, including 
error- correcting codes,'^''^ image restoration,'^'^ neural networks,'^ combinatorial prob- 
lems and so on. 

Although only the limit value (1/A^) (In 2') = lim„_^o (^{Z"-)^^^ — ij /n is usually em- 
phasized, RM can be considered to be a systematic procedure for calculating generalized 
moments (2'") of the partition function Z in the case of — * oo when Z depends on a 
certain external randomness. Here, characterizes the system size, n G R (or C) is a real 
(or comple?^) number and (■ ■ ■ ) means the average over the external randomness. For most 
problems, a direct assessment of {Z"^) is difficult for a general n G R (or C), whereas such an 
assessment for natural numbers n = 1, 2, ... is possible in the thermodynamic limit N —>■ oo. 
Therefore, (Z") is first computed for natural numbers and their analytic continuation is used 
to extend {Z"^) to n G R (or C). This is usually termed the replica trick. 

However, the validity of the replica trick is doubtful. The most obvious analytic con- 
tinuation, obtained under the replica symmetric (RS) ansatz, sometimes leads to the wrong 
results. The causes of these errors were actively debated in the 1970s until Parisi discovered 
the replica-symmetry-breaking (RSB) scheme for constructing reasonable solutions within 
the framework of RM.I^ Since this discovery, there have been no known examples for which 
physically wrong results have been derived by RM, in conjunction with the Parisi scheme if 
necessary. Therefore, RM is now empirically recognized as a reliable procedure in physics, 
although the mathematical justification of the replica trick still remains open. However, 
this problem is now generating interest again, in particular, in the application of RM to IP 
problems. Ths is because most theories in IP research have conventionally been developed 
with mathematical soundness highly valued.^'^ 

The purpose of this paper is to provide a method to approach the problems of RM. 
Specifically, we give a useful formula to compute (Z^) directly for n G C at finite N for a 
simple spin glass model, termed the discrete random energy model (DREM).^''^''^ This 
formula is numerically tractable, so one can directly observe how the system approaches the 
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thermodynamic limit with the aid of numerical calculation. Furthermore, this analytically 
clarifies the correct behavior for — oo, making a direct examination of the validity of 
RM. 

We have two main reasons for picking DREM from among the various disordered systems. 
First, this model is simple enough to handle analytically. It is already known that RM, in 
conjunction with the Parisi scheme, can evaluate the correct free energy for a family of 
random energy models (REM) including DREM at the limit of n 0.^^' However, the 
existing procedure seems at odds with a theorem concerning analytic continuation provided 
by Carlson,!^^ which holds for DREM of finite claiming the uniqueness of analytic 
continuation from natural numbers n G N to complex numbers ri G C, when the temperature 
is sufficiently low. For this, our approach shows that a phase transition occurs at a certain 
critical replica number rif. G [0, 1] in such cases, which clarifies that RM can be consistent 
with Carlson's theorem. This may offer a useful discipline to perform analytic continuation 
from n G N to n G R (or C) in RM. The second reason is a relationship between REM and 
certain problems of IP. Recent research on error-correcting codes has revealed that REM is 
closely related to a randomly constructed code.'^'^ These codes are known to provide the 
best error correction performance in information theory, and the performance evaluation of 
such codes is similar to the computation of (Z") for n G R^^' (see appendix^. Therefore, 
the current investigation will indirectly justify the RM-based analysis of error- correcting 
codes performed previously.'^''^''^ 

This paper is organized as follows. In section |21 we introduce DREM and briefly review 
how RM has been employed in conventional analysis of this system. Referring to Carlson's 
theorem, we address how the conventional scenario for taking a limit n — > seems contro- 
versial. In order to resolve this difficulty, we propose in section |21 a new scheme to directly 
evaluate (2'") for REM of finite and complex n without using the replica trick. Taking 
the limit oo, we analytically clarify how {Z"-) behaves in the thermodynamic limit and 
numerically verify this behavior. In section 0] we show how RM can be consistent with the 
results obtained by the proposed scheme. Section |S1 is a summary. 

§2. Replica method in the discrete random energy model (DREM) 

In order to clearly state the problem addressed in this paper, we first review how RM has 
been conventionally employed in analyzing REM.^''^ For convenience in the later analysis, 
we mainly concentrate on DREM, but the addressed problem is widely shared with other 
versions of REM as well. 

A DREM is composed of 2^ states, the energies of which, {A = 1,2, .. . ,2^), are 
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independently drawn from an identical distribution 

over M + 1 energy levels Ei = -M/2, -M/2 + 1, . . . , M/2 - 1, M/2. For each realization 
{e^}, the partition function 

2N 

Z = J2^M-P^a): (2-2) 



A=l 



and the free energy (density) 



kT 

F^-—\ogZ (2-3) 



can be used for computing various thermal averages. However, when the configurational 
average is required, one has to compute the averaged free energy (F) — (log^), the 
direct evaluation of which is generally difficult. Here, (• • • ) denotes the configurational 
average with respect to {e^}. On the other hand, the moment of the partition function (Z") 

can be easily calculated in various models for natural numbers n — 1,2, Therefore, the 

replica method evaluates the averaged free energy using the replica trick 

analytically continuting the expression of {Z"-) for n = 1,2,... to that for real (or complex) 
numbers n. 

For a given natural number n, the moment of DREM is calculated as 

2iv 2^ 2^ M M M 



z"> = E E • ■ • E E ^' K') En^n-Ep 



Ai=lA2=l A„=lii=0 ~ ' 12=0 ~ ' i2JV=0 
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N 



B=l ^1=1 



2™ 2™ 2™ 2 
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EE En 



Ai=l A2=l A„=l B=l 
2^ 2^ 2^ 2^^ / / n \ \ 

= E E ■ ■ ■ E n r ^ 5] ^^^^ (2-5) 

Ai=lA2=l A„=1B=1 \ \ fj,=l // 

where a = M/N, the function I{x) is defined as 

7(x)=log(cosh|). (2-6) 
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An identity 115=1 ^xp f X]|^=i ^ba^ j = exp (— was employed for obtaining the final 
expression of Eq. ()2-5j) . From now on, we focus on the case a > 1, for which the replica 
symmetry can be broken when the temperature is sufficiently low. 

Unfortunately, performing the summation in Eq. ()2-5|) exactly is difficult. Instead, in 
conventional RM, Eq. ()2-5j) is represented by the most dominant contribution in the sum- 
mation. This can be justified for natural numbers n in the limit N —>■ oo. Notice that the 
summation is invariant with respect to the permutation of the replica indices fi = 1,2, ... ,n. 
This replica symmetry restricts the candidates of the most dominant contribution to three 
possibilities, which are here referred to as solutions of the replica symmetric 1 (RSI), the 
replica symmetric 2 (RS2), and the 1-step replica symmetry breaking (IRSB). 

• RSI 

In RSI, all of the n replicas are assumed to occupy n different states B{= 1,2,..., 2^). 
Therefore, for a given B, 

E" j 1 (when B is one of the n occupied states), 

1^ U (otherwise). 

The number of ways to assign n replicas to n out of 2^ different states is 

2^ X (2^ - 1) X ■ ■ • X (2^ - n + 1) ~ 2"^. (2-8) 

For each case, the configuration of this type contributes 

exp{nNaI{p)), (2-9) 

in Eq. fl2-5|) . This means that the contribution to the moment from RSI becomes 

(Z") = 2"^exp(nAraJ(/5)) 

= exp niV (log2 + a log (cosh- )). (2-10) 



RS2 

In RS2, all of the n replicas are assumed to occupy a particular state B. Therefore, 
for a given B, 

E" In (when B is the occupied state), 

^^^^ = 1 n ^ fi. • ^ 
^^-^ U (otherwise). 

The number of ways to choose one out of 2^ states is 2^ . For each case, the configu- 
ration of this type contributes 

exp{NaI{np)), (2-12) 
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in Eq. ()2-5|l . which indicates that the contribution from RS2 is 

(Z") = 2^exp {nNaI{n(3)) 

= expAr|^log2 + alog(^cosh^^^. (2-13) 

IRSB 

In IRSB, n rephcas are assumed to be equally assigned to n/m states B, where m is 
an aliquot of n. Therefore, for a given B, 

E" I m (when B is one of the n/m occupied states), 

^^-^ (otherwise). 

The number of ways to select n/m out of 2^ states equally assigning n replicas to the 
n/m states is 

(2^)! n\ , , 

^ ' X -, 2™^. 2-15 



(2^ -n/m)! m"/™ 
For each case, the configuration of this type contributes 

exp Ara/(m/3)V (2-16) 
\m / 

in Eq. ()2-5p . Taking all the possibile values of m into account, the contribution from 
IRSB can be summarized as 

(Z") = y 2^^exp ( -Nal{ml3)] 
^ — ^ \m / 

m 

= y exp (—N [log 2 + aI {m(3)]) 
^-^ \m / 

m 

~exp fextr I— [log 2 + 0/(771/3)])) . (2-17) 

In the last expression, we have replaced the summation over 772, with the extremization 
with respect to 777 (extrm{- ■ ■ }), which is hopefully valid for large A^, analytically 
continuing the expression with respect to m from natural numbers to real numbers. 
The extremization with respect to m yields the condition 

log 2 + aJ (777/3) =0777/3/' (777/3), (2-18) 

which implies that the moment is expressed as 

(Z") = exp {nNa^V (m^/?)] , (2-19) 
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Fig. 1. The configuration of replicas for each solution. In RSI, n replicas are distributed in different 
states. In RS2, n replicas are concentrated in one state. In IRSB, n replicas are equally 
assigned to n/m states. For IRSB, however, this figure does not directly correspond to the 
solution because the critical value of m is not neccesarily an integer and can be larger than n 
for P > Pc. 



where rric is the solution of Eq. ()2-18|) . Eq. ()2-6|) indicates that rric can be represented 
as nic = Pc/ P, where the critical inverse tempereture /3c > is determined by 



log 2 + a ( log ( cosh y j ~ y t^nh y 



This provides a simple expression of the IRSB solution as 



0. 



(2-20) 



(Z") = exp (nNapriPc)) 

fnNa(3 , (3^ 
exp I — ^ — tanh — 



(2-21) 



The configurations of the replicas assumed for RSI, RS2, and IRSB are pictorially pre- 
sented in Fig. m It should be emphasized here that the above three solutions are derived for 
n = 1,2, . . ., assuming is sufficiently large. However, the obtained expressions are likely 
to hold for real n as well. Therefore, in conventional analysis, the replica trick of Eq. (j2-4p 
is carried out, selecting one possibly relevant solution of the three, which is hopefully valid 
for large N . 

The existing prescription for selecting the relevant solution is as follows?^ For small n, 
RSI, RS2, and IRSB are ordered RS2 > RSI > IRSB from the viewpoint of their amplitudes 
(Fig. 121). The contribution from RS2, however, converges to 2^ rather than unity for n ^ 0, 
and therefore, the replica trick leads to divergence. Hence, this solution is discarded. After 
excluding this solution, the leading contribution always comes from RSI, which guarantees 
a finite limit in Eq. ()2-4p . 

Actually, the answer obtained from this solution is correct in the case of high temperatures 
Q < 13 < Pc- However, this solution becomes invalid for low temperatures (3 > (3c, for which 
the correct answer is provided by IRSB. It may be worth noting that the value of rric, in 
this low temperature case, is placed in the interval n < rUc < 1 when n ^ 0, which is out 
of the ordinary range 1 < rric < n for n = 1,2,.... This prescription for taking the n — *• 
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limit is empirically justified for a family of REMs because it reproduces the correct answers 
obtainable by other schemes at the limit of n — > 0.^^' However, the following two issues still 
need further investigation. 

The first issue is the reason for expurgating RS2. Carlson's theorem, which guarantees 
the uniqueness of analytic continuation from natural ri G N to complex n G C, might 
be useful for resolving this problem.'^''^ Unlike other systems such as the Sherrington- 
Kirkpatrick modeP and the original REM,^ a modified moment of the current DREM 



-Af/3/2^^n^i/^^ which is extended from n G N to n G C, satisfies an inequality. 



l/N 



< 



< 



Re(n)^ 1/^ 

J]exp [-(3{e^ + M/2)] 

Re(n) 

^ 1 I ) = 2^"(") < O (exp 

.A=l 



(2-22) 



for Re(n) > and VA^ = 1, 2, . . ., since + M/2 is lower bounded by 0. Suppose that we 
could construct another extention tp{n]N) {n G C), which satisfies the the growth condi- 
tion, ipln; N) < O (exp [vr|n|]) *', and agrees with ((e-^^/2^)")^^^ at all the natural num- 

l/N 



bers n = 1, 2 

mn;N)\ 4 
V(n;iV)- 



This indicates that a similar inequality 



l/N 

ij{n; N) 



-Mf3/2 ^'jny 



< 



^^-Mf3/2Z)^) 
-M/3/2 ^\n\l/^ 



1/Af 



and ^(e ^'^^I'^ZY'^^^ completely coincide for n G N. Then, Carlson's theorem (Theorem 
5.81 in page 186 of Ref. |221)) ensures that \i}{n;N) - ((e-*W2^)"^VAf I j^g^tical to 0, 

implying that ilj{n; N) and (^(e'^^^^'^Z)'^^^^ are identical and, therefore, analytic continua- 
tion of (^{eT^^I'^ZY'')^^^ from natural n G N to complex n G C can be uniquely determined, 
unless analyticity of l^{er^'^^l'^ZY''Y^^ is lost in the limit N ^ oo. Since e"*^^/^ is a non- 
vanishing constant, this means that analytic continuation of the moment {Z'^)^^^ is also 
unique as long as [Z"^)^^^ remains analytic with respect to n in the limit oo. Hence, 

the dominant solution for n = 1, 2, ... , rather than for < n < 1, should be selected as the 
relevant solution for n — > 0. This recipe succesfully reproduces the correct answer for the 
high temperature region < /? < /?c- However, this is still not fully satisfactory because RS2 
becomes dominant for = 1, 2, . . . in the case oi (3 > (3c and, therefore, should be selected as 

*' This condition is necessary to exclude a trivial multiplicity caused by addition of certain analytic 
functions which vanish at all the natural numbers n = 1,2,..., such as sin(7rn). 



< O (exp[7r|n|]) holds for Re(n) > and the difference 
vanishes at all the natural numbers n = 1, 2, . . ., as %l){n\ N) 
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Fig. 2. Solutions obtained under the RSI, RS2 and IRSB assumptions. RSI and RS2 agree at 
n = 1 while RS2 contacts IRSB at a certain point of positive n. (a) For (i < (3c, the contact 
point is placed in the region of n > 1. (b) For (3 > (3c, on the other hand, it is located in 
< n < 1. 



the relevant solution for n — if analyticity is preserved for N ^ oo, which, unfortunately, 
leads to a wrong answer. Hence, a certain phase transition must occur at a critical replica 
number < Hc < 1 for (3 > (3c- However, to the knowledge of the authors, such a phase tran- 
sition with respect to n has not yet been fully examined for most disordered systems.'^''^ 
This might be because so far the greatest attention has been paid only to the final results 
at the limit n ^ 0. However, detailed analysis of phase transitions of this type may soon 
be needed as the replica calculation for non-vanishing n has recently begun to be employed 
in problems related to IpEOl-IM analysis of certain dynamics involved with multiple time 
scales.!^ 

The second question is the origin of IRSB. In conventional analysis at the limit n — > 
gQUJSnjElEHJ ^j-^jg solution is introduced by modifying RSI to keep the entropy of the correct 
solution non-negative for (3 > (3c- It would appear that IRSB originates from RSI. However, 
at least for positive n, this association seems unlikely because the two solutions cross each 
other only at = (see Fig. |21). Therefore, it is impossible to relate the origin of IRSB 
to RSI at large n, from which the solutions of smaller n are extrapolated. On the other 
hand, IRSB contacts RS2 at a certain point of positive n, implying that IRSB bifurcates 
from RS2. As the contact point is located in < n < 1 for /5 > /3c, this seems consistent 
with the aforementioned possible phase trainsition at n = (see Fig. 121(b)). Nevertheless, 
such a scenario cannot be so easily accepted since RS2 still dominates IRSB even below the 
contact point; a mere contact does not change the dominance between the two solutions. 

It may help in resolving these problems to analyze DREM employing a completely dif- 
ferent methodology. In the next section, we provide a scheme to calculate the moment of 
DREM as a step towards clarifying the mysteries of RM. The proposed method is powerful 
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enough to evaluate the moment in the right half of the complex plane Re(n) > for any 
arbitrary finite without using the replica trick, which supplies a model answer for RM. 

§3. Direct calculation of the moment for grand canonical DREM (GCDREM) 

3.1. General formula 

In order to introduce a novel scheme for caluculating moments of the partition function 
of DREM, we first rewrite the partition function using the occupation numbers {i = 
0,1,2,...,M) as 

2'^ M M 

Z = '^exp{-l3eA) ='^niexp{-(3Ei) = u'^'^niUj' (tu = e"^). (3-1) 

A=l i=0 1=0 

This means that the partition function of DREM can be completely determined by a set of 
occupation numbers {ui}, in which details of the energy configuration {e^} are ignored. This 
makes it possible to assess the moments {Z"-) directly from {nj} without referring to the full 
energy configuration {e^}. This significantly reduces the necessary cost for computing the 
partition function when the calculation is numerically performed. 

Two methods are known for generating {rii}. The straightforward method is to count 
Hi independently, drawing the 2^ energy states from Eq. ()2-l|) . The system obtained by 
this is referred to as the canonical discrete random energy model (CDREM). Although this 
yields a rigorously correct realization of DREM that satisfies the constraint YlfLo ^« ~ 2^, 
it takes 2^ steps to count {ui} and hence, is computationally difficult. In order to resolve 
this difficulty in numerical experiments, Moukarzel and PargsP^''^ have proposed the grand 
canonical version of the discrete random energy model (GCDREM) *). In GCDREM, the 
occupation numbers are independently determined using the Poisson distribution 

P(n.)=e-^'^, (7, = 2^P(i?,)), (3-2) 
nil 

where ji is the average occupation number. The greatest advantage of GCDREM is that 

one can drastically reduce the necessary computational cost for generating {ui} from 2^ 

to M + 1. One possible drawback is that the constraint YlfLoni = 2^ is only satisfied 

in average, ^^^q'^*^ ~ which implies that this method does not correspond strictly 

to the original model. However, the RM-based calculation indicates that thermodynamical 

properties of GCDREM become identical to those of CDREM for oo, which is provided 

in appendix El and one can show that the difference rapidly vanishes as A^ becomes large 

*' Employment of GCDREM is not essential for reducing the numerical cost. We have discovered a 
scheme for generating CDREM in a time scale similar to that for GCDREM. It is shown in appendix [UI 
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and is almost indistinguishable even for = 3, verified in appendix O In addition, this 
version of DREM has another advantage in analytic calculations as the summation can be 
carried out independently, as is shown below. 
In GCDREM, the moment is expressed as 



oo oo 



no=Oni=0 nM=0 

oo oo oo 

= (^-"^11111^ ^ ■■■ ^ PK)P(r2i)---P(M (E^^^' + ^l ' (3-3) 



no=Oni=0 nM=0 \ i=0 

where an infinitesimal constant e > is introduced in order to keep Z positive even when all 
the occupation numbers vanish. This makes it possible to employ an identity for the positive 
number c 

r{-n) 

where the integration contour is shown in Fig. El for assessing the moment as 

TlAf 

ip 



" „ / oo \ / oo 

Jim jj^-p)—^e-^P ( ^ P{n,)e---A ( P{m)e 



oo 



oo 

PK/)e~"«-"M dp. (3-5) 
It is worth noting that the summation in this expression can be independently carried out 



as 



oo oo ^ 

ni=0 ni=0 * 

Therefore, the moment can be summarized as 

nM M 

in = lim / i-p)—' exp [-ep - - e^'ni^P- (3-7) 

Since this is convergent for Re(n) > 0, the following expression gives the analytic continua- 
tion of the moment to the right half complex plane of n, 

nM M 

= 1^ / (-P)— ^ exp [- 5^(1 - e-'nl^]dp. (3-8) 
Pi-n) Jh ^ 
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Fig. 3. The integration contours. 



3.2. Thermodynamic limit 

Using Eq. ()3-8p . one can analytically examine the behavior of the moment in the thermo- 
dynamic limit iV, M — i> oo, keeping a = M/N finite. For this, we first convert the contour 
integration in the expression to an integration on the real axis for p — 1 < Re(n) < p, where 
p is an arbitrary natural number. Further, we define a function 



M 



f{p) ^ exp [- 5^(1 - e-'07.] = E f^P' 

j=0 



(3-9) 



i=0 



and a series of truncated summations as 



(3-10) 



j=0 



which satisfy the identity 



H 



(3-11) 



H+I 



for p — \ < Re(n), since the contribution from the contour I vanishes. Using this identity, 
the moment can be rewritten as 



nM 
UJ 2 

ri-n) 



(3-12) 



H 



Eq. ()3-lHl guarantees that the infrared divergence is removed for Iie{n) < p in this expression. 
Therefore, the moment can be rewritten for p — 1 < Re(n) < p as 



in 



UJ 2 

r(-n) 



p-'-Vip) - f^'\p)]dp 



(3-13) 



where the function F is replaced by the ordinary gamma function F. 
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As we have a particular interest in the case of p = 1, i.e. < Re(n) < 1, let us focus on 
the behavior of the following expression, 



r{-n) Jo 

M 



r -n) 



p 



--i[exp{-5^(l-e-->)7.}-l]rfp. 



(3-14) 



1=0 



To examine the behavior of the thermodynamic limit, it is convenient to introduce new 
variables 



X 



y 



Inp, 



Na " Nap 

instead of i and p, yielding the following expression for the moment 



(3-15) 



UJ 2 



r(-n) 



y, 



(3-16) 



where 



S{y) 

n{x,y) 



-n«/?y + llog(l-e-^(^)^ 



(3-17) 



and 



1 - «) log 2 + aH{x) + — log(l - e" 



H(x) = — xlogx — (1 — x) logfl — x). 



(3-18) 



Here, we have replaced the summation with an integration, which can be verified when both 
M and are sufficiently large. 
For further analysis, the identity 



aiu) = ^log (l -e" 



{u > 0) 
u {u <0) 
u9{-u) 



(3-19) 



which holds for large N, may be useful. The shape of this function is displayed in Fig. EJ 
As this becomes singular at m = 0, the phase transition may occur in n space for A^ — > oo 
We examine this below. 
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g(u). 







u 



Fig. 4. The shape of g{u) for N ^ oo. This is not analytic at n = 0. 

Eq. ()3-19p means that the function Ti. can be expressed as 

n{x, y) = {1- a) log2 + aH{x) + af3{y - x)d{y - x) 



(3-20) 



in the thermodynamic hmit. As a function of x, this exhibits three behaviors depending on 
a given value of as shown in FigjSl Employing the saddle point method, the maximum of 
1-L{x, y) given y provides J^{y) for large iV, which yields 

1/1 
2 2 



lim — log J^{y) = < 

N^oo iV 



^1 — a) log 2 + aH{x) {xc<y< 



(3-21) 



^ (1 - a) log 2 + aH{l3) + ajSy {y < Xc) , 

where solution of 

H'{x,) = /3, (3-22) 

the behavior of which is shown in Fig. |B1 Here, the function H is the Legendre transformation 
of the function H, 



The function Q becomes 



i/(/3)=log(2cosh^] 



g{y) = -nal3y + - log^(y)0(-^(y)) 



(3-23) 



(3-24) 



for large A^, which directly controls the behavior of the moment in Eq. (|3T6|) . The behavior 
strongly depends on the relation between Xc and x* = (l — tanh /2 which satisfies the 
condition 



[l-a)\og2 + aH(x*) = 0. 



(3-25) 
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^(x,y) 




Fig. 5. The schematic shape of Ti.{x, y) for N — oo. It depends on the value of y. 



r(y) 




6. The schematic shape of for N ^ oo obtained from the maximal value of Ti.{x,y). 



(A) X* < Xc 

Since Xc and x* are defined in Eq. (j3-22|) and Eq. (j3-25|) . respectively, the condition 
X* < Xr can be written as 



[l-a)\og2 + aH{xc) > 0. 



(3-26) 



This is satisfied for (3 < (3^, i-e. the high temperature phase. Notice that for a < 1, as 
Eq. (|2-2(Jj) does not have a positive solution, this is always satisfied independently of 
p. Then, Eq. (j3-24j) can be represented as 

g{y) = -na(3y + [(1 - a) log2 + aH{P) + a(3y]e{y, - y), (3-27) 

which is shown in Fig. [7| Here, yc is defined by Tiyc) = (Fig. IHI), which yields 

Vc = -^[(1 - a) log 2 + aHm. (3-28) 



The moment is then calculated as 

nM r 



(Z") = -Na(3 



UJ 2 

ri-n) 



Vc 



,a0nNa-yc) 



+ 
1 



exp (1 — a) log2 + aH{(3) + a[3{\ — n)ydy 
exp {—nal3y)dy 



+ 



n 



r{l-n) r{2-n)_ 



(3-29) 
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Fig. 7. The schematic figure of G{y)- The maximal value determines the behavior of {Z"'). 



Therefore, the asymptotic behavior of the moment is 

hm llog(Z") = «/5nivfl-y, 

N-*oo iV \Z 

= n ^log2 + alogcosh (3-30) 

in this phase. This is consistent with RSI, as can be seen from Eq. ()2TUp . 

(B) X* > Xc 

The condition x* > Xc is satisfied for (3 > jSc, which may correspond to the low 
temperature phase. The function Q{y) has the form 

g{y) = ~na(3y + [(1 - a) log2 + aH{y)]e{x* - y), (3-31) 

the behavior of which is further classified into two cases depending on the replica 
number n. 

- (i) H\x*) > n(3 

Since x* is located in Xc < x* < ^, there exists a critical replica number < ric < 1, 
which is characterized by 

nc ^ -^H'{x*) = |. (3-32) 

The condition H'{x*) > n(3 is satisfied for n < ric- The function Q{y)., shown in 
Fig. [3 is maximized aX y = x* . Therefore, the moment can be expressed as 

lim ^ log (Z") = na(3 {]: - x' 

= ^^ tanh^. (3-33) 

This behavior is identical to that of IRSB predicted by RM, as can be seen in 
Eq. (EUD. 
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Fig. 8. The behavior of the moment in the thermodynamic Hmit obtained from the exact expression 
(full lines). In the high temperature phase P < (5c, the behavior of the moment is simple and 
corresponds to the result obtained from RSI. In the low temperature phase (3 > f3c, the 
behavior of the moment has interesting properties. The moment corresponds to that obtained 
from RS2 for n > nc = (3 / (3c and corresponds to that obtained from IRSB for n < Uc- A phase 
transition occurs at n = Uc- 



- (ii) H\x*) < n(3 

The condition H'{x*) < n(3 is satisfied for n > ric- The function Q{y) is maximized 
not at y = X* but a.t y = yc- Therefore, the moment can be asymptotically 
expressed as 

1 n3 
lim — log (Z") = log 2 + a log cosh (3-34) 

This coincides with the behavior of RS2 obtained by RM, as can be seen in 
Eq. (jni. 

The results are summarized in Fig. |H1 In the high temperature phase (3 < (3c, the behavior 
of the moment is simple, being expressed by RSI of RM. In the low temperature phase /? > Pc, 
the behavior of the moment has two possibilities depending on n. More specifically, in the 
limit oo, the moment approaches RS2 for n > whereas IRSB represents the correct 
behavior for n < Uc- This means that there exists a phase transition in the space of replica 
number at n = Uc- In conclusion, these are consistent with the known results obtained by 
RM at the limit n QUBmm 

3.3. Numerical validation 

Eq. (j3-8|) is formulated as a two dimensional summation with respect to p G C and 
i = 0, 1, ... , M, which is numerically tractable. This means that one can utilize Eq. (13 -8^ 
or Eq. ()3-16|) to numerically examine the behavior of DREM for a finite system size N, and 
how fast the results obtained for N ^ oo become relevant as grows large. 
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Fig. 9. The logarithm of {Z"^) calculated from Eq. (|3-16|) and data obtained from numerical ex- 
periments of Eq. using 10, 1000 and 100000 samples. The system size is = 10. The 
points indicated as "Exact" are the results obtained from our expression Eq. ()3-16|) . 

Fig. El shows the logarithm of {Z"") calculated by Eq. (j3-16|) and {Z"^) numerically eval- 
uated from 10, 1000 and 100000 experiments, with = 10. One can see that the data 
from the numerical experiments converge to the results of Eq. ()3-16|) . This verifies that our 
expression accurately provides the moment even for a finite system size. 

We next compare the results from our expression and RM in Fig. ^1 At high temper- 
atures, our result is consistent with RSI as expected. The difference is negligible for all of 
the range < n < 1 even at = 10. At low temperatures, our result fits RS2 for larger 
n > ric, while IRSB exhibits excellent consistency for smaller n < ric- There is a slight 
difference between our expression and IRSB for A^ = 10. The difference, however, becomes 
indistinguishable for A^ = 100. This strongly indicates that there occurs a phase transition 
between RS2 and IRSB at n = n^- in the limit A^ oo. 



§4. Origin of IRSB: Extreme value statistics 

The preceding two sections indicate that DREM exhibits a phase transition with re- 
spect to the replica number n at a certain critical point ric G [0, 1] when the temperature 
is sufficiently low. In this section, we discuss how this transition can be understood in 
the framework of RM. A formalism previously introduced for examining the domain size 
distribution of multi-layer perceptrons is useful for this purpose.!^ 

Since the partition function of DREM typically scales exponentially with respect to A^, 
we first express this dependence as 

1' 



exp 



-Nap [y-- 



(4-1) 



where y — \ represents the free energy normalized by the scale of the energy amplitude M 
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[p < (3 ] 
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[(3 > (3,] 




0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 



n 

Fig. 10. The logarithm of (.Z'") obtained from our expression and RM. The points indicated as 
"Exact" are the results obtained from our expression Eq. ()3-16|) . The lines indicated "RSI", 
"RS2", and "IRSB" are the results obtained from RM, using Eq. XTm . Eq. (I^TH . and 
Eq. (|2-21j) . respectively. At high temperatures f3 = /?c/3 < Pc, our result is consistent with 
RSI. The difference is very small even for = 10. At low temperatures f3 = 3/3c > /3, our 
result fits RS2 for larger n > ric = (3c/ f3 = 1/3 while IRSB provides good consistency for 
smaller n < Uc- There is a slight difference between the results from our expression and that 
from IRSB at A = 10. The difference, however, becomes imperceptible for A^ = 100. 

for a given realization {ba} {A = 1, 2, ... , 2^) (or {rii} {i = 0, 1, ... , M)). Clearly, y is a 
random variable. Let us assume that the probability distribution of y, V{y), scales as 

Viy) ~ exp [-Nc{y)] , (4-2) 

where c{y) ~ 0(1) for large A^. Notice that an inequality 

c{y) > 0, (4-3) 
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must hold to make V{y) satisfy the normahzation condition j dyV{y) = 1 for oo. Eq. 
indicates that the moment of the partition function can be calculated as 



{Z^) = j dyexp 



-Nnap [y-^ 



exp 



y 



N extr I —na(3 [y ) — c{y) 



2 



ny) 

(4-4) 



This formula, however, may not be useful for computing (2'"), as directly assessing c{y) is 
rather difficult. Instead, it can be utilized to evaluate c{y) from (Z"), since Eq. ()4-4p implies 
that c{y) can be obtained from 

c{y) = extr _ _ 1 h, , (4-5) 

where (1/A^) In (2'"') can be computed by RM for any real number n. This, in conjunction 
with the normalization constraint in Eq. ()4-3|) . offers a useful clue to identify the origin of 
the phase transition with the aid of RM. 

For (3 > (3ci RS2 provides the dominant solution for natural numbers n = 1,2,... in 
the thermodynamic limt. Carlson's theorem implies that this should offer a unique analytic 
continuation if no phase transition occurs. Therefore, let us first insert this solution, from 
Eq. (jHSl), into Eq. (g]!)), giving 

c{y) = extr < —na(3 ( 1/ — 77 ) — log 2 — a log (cosh ^ 



2J ° ° V 2 

= {a- l)\og2- aH{y). (4-6) 

Notice that this does not satisfy the necessary condition given in Eq. ()4-3p for y > x*, and 
the critical value y = x* corresponds to n = ric = H'{x*)/(3 = (3c/ (3, which signals the 
occurrence of a phase transition dX n = ric within the framework of RM. 

The following consideration about the energy configuration provides a plausible scenario 
for this transition. As have already been pointed out,-^ the partition function Z in the 
low temperature region (3 > f3c, can be considered to be dominated by only the minimum 
energy emin of a given energy configuration {e^}, such as Z ~ exp [— /3emin]. This implies that 
V{y) of the normalized free energy y — ^ = ~{^ogZ)/{Nal3) is given by the distribution of 
^min/M under the assumption that each energy level {A = 1,2, ... , 2^) is independently 
drawn from an identical distribution, given in Eq. ()2-Hl . which has been already examined 
in the research of extreme value statistics (EVS).'^ In the current case, the average occu- 
pation number of an energy level Ej = i — = M fa; — 7,- = 2^~^^ | 

2 ^ y M/2 + Ei 

exp A^ ((1 — a) log 2 + aH{x)) {i = 0,1, ... ,M and x = i/M G [0, 1]), grows exponentially 
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large with respect to for x > x*. This indicates that V{y) is very small for y > x* because 
X > X* does not provide ej^in/M for a given energy configuration {e^} except for very rare 
cases, as energy levels lower than Ei = M (^x — ^) are included in the configuration with a 
very high probability. On the other hand, 7^ becomes exponentially small for < x < x*, 
which means that the probability of having an energy level labelled by x of this interval 
in the energy configuration is low. Therefore, V{y) also becomes small for < y < x*. 
However, the functional form of V{y) is not symmetric between y > x* and < y < x*. 
Actually, detailed analysis of EV^^'I^ shows that V{y) exhibits the asymmetric scalings 



p. ^ , exp[-exp[-Ar((a-l)log2-aif(y))]], {x*<y), 
^^'^^ exp[-N{{a-l)log2-aH{y))], {0 < y < x*), 



for large A^, which provide a singularity at y = x*. It may be worth noting that 1 — 
exp [— JF(?/)], which comes out in Eq. ()3-16|l after inserting Eq. ()3-17|l in the analysis presented 
sectional corresponds to the cumulative distribution J^^dyV{y). Eq. ()4-7|) implies that the 
extremum point in Eq. ()4-4|l for large n{> ric) is in < y < x*, which leads to RS2 and 
consistently provides the exponent of Eq. ()4-(i|l . On the other hand, the constant y = x* offers 
the extremum for all small n{< ric), which corresponds to IRSB in the framework of RM. 
The intuitive implication of this is that {Z"') is dominated by an atypically low minimum 
energy generated with a small probability for n > ric, whereas the typical minimum energy 
emin = M (x* — I) provides the major contribution to {Z'^) for n < ric- Thus, the origin of 
the phase transition can be attributed to the singularity of the free energy distribution V{y), 
exhibited in the case of low temperatures /3 > Pc- 

Our analysis also illustrates that Carlson's theorem is not necessarily useful for validating 
RM because an analytic continuation given for finite A^ can exhibit a singularity in the limit 
of A^ ^ 00 and, therefore, taking the limit A^ ^ 00 prior to n — > for determining the 
continuation on the basis of expressions for n = 1,2,..., which is usually performed in RM, 
sometimes yields a wrong answer for n < 1. However, in the current system, this drawback 
can be resolved by taking a constraint for the free energy distribution (i.e. Eq. (j4-3|) ) into 
account, which leads to the conventional IRSB. Although the importance of the notion 
of EVS in RM has been already addressed at the limit n 0,^^' to our knowledge, the 
current analysis is the first work that directly clarifies how the property of EVS relates to a 
corruption of the analytic continuation in RM, expressed as a phase transition with respect 
to the replica number n. 
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§5. Summary 



In summary, we have offered an exact expression, Eq. (j3-8j) . of the moment of the partition 
function {Z"^) for GCDREM, without employing the rephca trick, which is vahd for any 
arbitrary system size and complex number n (Re(n) > 0). Simplifying the expression for 
< n < 1, we have shown that a phase transition with respect to the number of replicas n 
occurs at a certain critical number G [0, 1] in the thermodynamic limit N ^ oo when the 
temperature (3~^ is sufficiently low, if the ratio a = M/N is greater than unity. This implies 
that Carlson's theorem, which guarantees the uniqueness of the analytic continuation from 
the expressions for n = 1, 2, ... to those for n G C, is not necessarily useful for validating the 
replica method, because taking the thermodynamic limit N oo prior to the continuation 
fails to derive a correct result when the analyticity is broken for n < 1 in the case of infinite A^. 
However, it has also been shown that this drawback can be overcome by taking the statistical 
property of the minimum energy level into account, clarifying how IRSB originates from a 
replica symmetric solution, which has been conventionally discarded, at the critical replica 
number n^. We hope that the results obtained here offer a useful insight to unlock the 
remaining mysteries of RM. 

Since Eq. (|3-8p is valid throughout the right half complex plane Re(n) > for any finite 
N, one can directly observe how the singularities of (Z^) approach the real axis of n as 
N oo.^'^' This is sometimes referred to as Lee- Yang's scenario of the phase transition.!^ 
Analysis along this direction is currently under way.l^ 

Acknowledgement 

Support from Grant-in- Aids of MEXT, Japan, No. 14084206 (YK) is acknowledged. 

Appendix A 

Link of DREM to error- correcting codes 

As was shown in Ref. IHI, certain types of spin glass models can be related to error- 
correcting codes. We here show that a known evaluation scheme of the error correcting 
ability of a random code ensemble can be reduced to a calculation of the general moment of 
the partition function with respect to DREM.'^''^ To the knowledge of authors, a remark 
on this relationship has already been made in Ref. EZj). 

In a general scenario of error correcting codes, an A^-dimensional binary vector s = 
{si, S2, . . . , sjy) (sj = 0,1, i = 1,2, . . . , N) is encoded to a codeword t = (ti, ^2, • • • , ^m) 
{ti = 0,1, i = 1,2, ... , M) of an M(> A^)-dimensional binary vector and transmitted via 
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a noisy channel. We here concentrate on a binary symmetric channel (BSC) in which each 
component of the codeword ti is independently flipped to the opposite letter in the alphabet 
(0 or 1) with a probability < p < 1/2. This implies that the vector r = t + n (mod 2) is 
received at the other terminal of the channel, where n is the noise vector, the component 
of which becomes 1 independently with probability p and otherwise. However, as the 
codeword is represented somewhat redundantly, decoding r provides the correct message s 
with a high probability for sufficiently small p. 

It is known that the performance of correctly retrieving s from r becomes good when 
a code C (i.e., an invertible map C : s ^ t) is randomly constructed,'^''^ providing a 
code ensemble. Let us evaluate the average probability of failing to correctly retrieve s in 
order to characterize the potential error correcting ability of the ensemble. We focus on 
the maximum likelihood (ML) decoding, which selects a codeword closest to the received 
vector r and returns a message vector corresponding to the codeword as an estimate s of s. 
ML decoding minimizes the decoding error probability Pe{C) when codeword vectors t are 
equally generated at the transmission terminal.'^ 

Notice that the message s can be correctly identified if its codeword t is provided, since 
the code is constructed as an invertible map. Therefore, for evaluating Pe{C), it is convenient 
to introduce an indicator function AML{t,f\C) that returns 1 if r is not correctly decoded 
to the correct codeword t and otherwise, which means the decoding error probability can 
be computed by 



where P{t\C) is the probability that the codeword vector t is generated given C and P{r\t) is 
the conditional probability that r is received when t is transmitted. P{C) is the probability 
that a code C is generated. X]£gc denotes a summation over 2^ codeword vectors given C. 

We assume that the code C is designed by the source coding technique such that t is 
uniformly generated as P{t\C) = \/2^ ^ In addition, for BSC, the conditional probability 
can be represented as 




(A-1) 



t,r 



and, hence, its average over the code ensemble can be represented as 



{PE{C)), = Y,nC) E PmP{r\t)Aui.{t,r\C) 



(A-2) 



tec.r 



P{r\t) 



exp y-FY^ti (I 

(2 cosh f)"' 



(A-3) 
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where 5x,y returns liix = y and 0, otherwise and F = log [(1 — p)/p]- 

Unfortunately, expressing Z\ml(^, rlC) in a rigorously treatable form is difficult. However, 
Gallager's inequality 



yt'(iC\t 

which holds for arbitrary n > 0, offers a good upperbound for this.'^''^ Here X]t'gc\t 
denotes a summation over 2^ — 1 codeword vectors t' of C excluding the possibility of the 
correct codeword t. Inserting this into Eq. ()A-2|) provides 



{PE{C)),<Y,P{C)Y.PmP^^{r\t){ J2 P^"{r\t) 
c t,r \t'ec\t 

'^P(C)^P^(r|0) I P^ir\t') 



c r 



,t'ec\0 



n=±i (2 cosh ^) c \t'^c\0 J 

where we have performed the gauge transformation t ^ 0, t' — t ^ t' and r — t ^ r, 
assuming any code C contains the zero codeword t = 0^ Minimizing the final expression 
with respect to n > 0, we can obtaine the tightest bound of the average decoding error 
probability. 

We here address that E(t) = J^fLi (| ~ "^^.tj Eq. ()A-5|) obeys Eq. ()2-H) independetly 
of r when each codeword t is equally generated in the ensemble such that each component is 
independently selected from an identical unbiased distribution P(tj = 1) = P(tj = 0) = 1/2, 
which holds for the random code ensemble,'^''^ although non-trivial correlations of 'energy' 
E{t) between 'states' t arise making the energy distribution different from Eq. ()2-l|) in the 
case of practical linear codes. Therefore, for the current random code ensemble, Eq. ()A-5|1 
can be simplified as 

/ / r F 1 \ \ 

(A-6) 



cosh 



A=l 



F 



1 + n 



where (■ ■ ■ ) denotes the configutational average with respect to the 'energy states' {A = 
1, 2, . . . , 2^ - 1) following Eq. (jH])- Regarding F/{l + n) as the inverse temperature, this 
means that an assessment of the average decoding error probability can be linked to calcu- 
lating the general moment of the 'partition function' Z = X]a=i^^^P ["TTn^^] DREM. 
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Appendix B 

The replica analysis of GCDREM 



Section |21 indicates that the generalized moment of partition function can be evaluated 
without using RM for GCDREM even when the system size is finite. However, for direct 
comparison to the conventional analysis, it may be helpful to demonstrate the conventional 
RM-based analysis as well. Therefore, we here provide a brief sketch of the replica calculation 
of GCDREM. 

From eq. ()3-Hl . we first obtain an expression 



M M M 



ii=0 12=0 in=0 \ai=1 
M M M M 

= E E ■ ■ ■ E n ((nF-''-)-'S:-"-) . (B-l) 

«i=0«2=0 i„=0 i=l 

for n = 1, 2, . . ., where (■ ■ ■ ) represents the average over the Poission distribution, Eq. (|3-2p . 
Following the conventional recipe of RM, let us next evaluate the candidates of the most 
dominant contribution in the limit N ^ oo under the RSI, RS2 and IRSB assumptions 
following the conventional scheme of RM. Before proceeding further, it may be worth men- 
tioning that the dynamical variables in GCDREM are not single states but energy levels 
i = 0, 1,2, ...,M, each of which is composed of multiple states. This difference makes 
composition of the solutions slightly different from that of CDREM provided in section |21 
although the final result is unchanged. 

• RSI 

In RSI, all the n replica levels are assumed to be allocated to n different levels i{= 
0, 1, 2, ... , M). Therefore, for a given level i, 

I 1 (when i is one of the n allocated energy levels), 
(otherwise). 

When i is an allocated level, 

(^nf"--'''"'^ = 7, ^ exp [N ((1 - a) log 2 + aHix))] , (B-3) 

and 

a;*^M=i^».'M = exp [-l3i] exp [-Nal3x] , (B-4) 
hold for large A^, yielding the contribution 

'^I:;=^s^..A^i^;^,5,,„^ ^ e^p[N{{l- a) log2 + a{H {x) - px))], (B-5) 
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where x = i/ M = i/{aN). This is maximized at Xc = (1 — tanh |) to 

exp ^log 2 + a log ^cosh — , (B-6) 

which represents the contribution from a certain rephca level i^{= 1,2, ... ,n). Con- 
tributions from other replicas become smaller than Eq. ()B-6|) as any two replica levels 
must be located at different levels under the current assumption. However, the differ- 
ence becomes negligible because there exist many levels in any vicinity of x = Xc in 
the limit N oo. Taking the pref actor lu"^ in front of the summation in Eq. ()B-1|) 
and the number of permutations over replica indices into account, this implies that the 
dominant contribution under the RSI ansatz is evaluated as 

(Z") = n! X expnA^ (log 2 + a log (cosh — 



exp nN ( log 2 + a log ( cosh ^ ) ) , (B-7) 



which is identical to the RSI solution of CDREM, Eq. ()2-10p . and equivalent to Eq. 

dnisnD- 

RS2 

In RS2, all the n replica levels are assumed to be allocated to a certain single level. 
Therefore, for a given level i, 

\ n (when i is the allocated energy level), 
^^-^ 1^ (otherwise). 

When i is the allocated level, 

^E;=,5^.^,\ { li ^exp[nN{{l-a)\og2 + aH{x))] {x>x*), 
/ [ 7i ^exp[iV((l -a)log2 + aiJ(x))] {x<x*), 

where x* is determined by Eq. ()3-25|) and 

u}''^^=i^'-'t' = exp [-n(3i] exp [-nNa(3x] , (B-10) 
hold for large A^. Notice that ( ''"^ ) behaves differently depending on whether 



X > X* or not. Therefore, the behavior of the dominant contribution obtained by 
maximizing (n^ '"'^ j ^«Ep=i'5i,iM is different depending on the relation between x* 
and the position of the maximum Xc- For /3 < /5c or a < 1, x^ = (1 — tanh|)/2 
becomes greater than x* , which yields the RSI solution, Eq. ()2-10|) . in spite that the 
RS2 ansatz is currently employed. This is because the RS2 ansatz for energy levels 
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does not necessarily mean that n replica states are identical, in particular, when the 
occupation number Ui is exponentially large, to which Xc > x* corresponds. In such 
cases, even if n replica states are allocated to an identical energy level, they are typically 
distributed to n different states placed at the energy level, which corresponds to the 
RSI ansatz of CDREM, and, therefore, Eq. ()2-10j) should be reproduced. On the other 

/ y~*^ S-\ 

hand, for (3 > /5c, (n^ '''i^ \ ^'T,^=lS^,^^, jg maximized at Xc = {I - tanh^)/2 < x*. 



2 

This provides 

(Z") = expN |^log2 + alog |^cosh^^^ , (B-11) 

which is identical to the RS2 solution of CDREM, Eq. (j2-13p . and equivalent to Eq. 
(j3-34j) . As Eq. (j2-13|) can be dominant at n = 1,2,... only for /3 > /3c, this is consistent 
with the result of the replica analysis of CDREM. 
IRSB 

In IRSB, n replica levels are assumed to be equally allocated to n/m levels. Therefore, 
for a given level i, 

E"" I m (when i is the n/m allocated energy levels), 

(otherwise). 

When i is the allocated level, 

^Y:;=rS,,,\^(-fr^exp[mN{{l-a)log2 + aH{x))] {x>x*), 
7i^exp[iV((l-a)log2 + a/7(x))] {x < x*), 



and 



= exp [-mPi] exp [-mNaPx] , (B-14) 



hold for large A^. Similarly for the case of RS2, this reproduces the RSI solution, Eq. 
(j2-l(Jj) . for /5 < /5c or a < 1. So, we focus on the low temperature phase /3 > Pc- Then, 



n^" ' " y cj^^'^^i '•''^ is maximized at Xc = (1 - tanh ^) /2 < x* to 

/, , / , md\ mad\ ,^ 
expiV Mog2 + alog f cosh^ j ^j, (B-15) 

which represents the contribution from one of the n/m allocated levels. The number 
of ways to select n/m out of M levels equally allocating n replicas is negligible for 
subsequent calculation. Taking contributions from all the n/m allocated levels and 
extremization with respect to m into account offers 

I \ ^ I ( mj3 

(Z") ~ exp extr < — log 2 + a log I cosh 

\ m \ m \ V 2 
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= exp Nna(3 ^- — x* 
= expiv(^^tanh^^ , (B-16) 

which is identical to the IRSB solution of CDREM, Eq. ()2-2H) . and equivalent to Eq. 
()3-33p . As Eq. ()2-21|) can be dominant n = 1,2, . . . only for P > [3c, this is consistent 
with the result of the replica analysis of CDREM. 

Appendix C 

Efficient sampling in CDREM 



We here show that one can sample {nj} with an 0{M) computational cost in CDREM 
as well as in GCDREM. According to the probability distribution for each level 

the probability to sample a configuration (no, ■ ■ ■ iUm) is 

V{n„ m, ■ ■ ■ , um) = {P{Eo)r{P{E,)r^ ■ ■ ■ {P(E^)}«M^ (C.2) 

To generate a sample (no,ni, ■ ■ ■ ,nM) in practice, we first determine no according to the 
probability 

V{no,arbztrary) = (po)"«(l - j9o)'"-"°— — ^ (C-3) 

no!(2^ - no)! 

{po ^ PiEo)). (C-4) 

We then determine ui according to the probability 

V{no;n,, arbitrary) = (p,y^^ (1 - p.f'' -o-r.^ (^-5) 

ni!(2^^ — no — ni)\ 



Repeating this procedure to n^-i as 

_ AT- , l\ I —1 / 

V{no, ni, ■ • • ; um-i, nu) = {pm-iT' (1 - Pm^i) 



nM-i!(2^-E2o'^«)! 



(C-7) 
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we obtain a set of (no,ni, ■ ■ ■ ^um)- This guarantees that the identity 

M 



(C-8) 



i=0 



holds, which characterizes CDREM. Similarly for the case of GCDREM, this can be per- 
formed in 0{M) computation. 

A comparison of numerically evaluated moments of the partition function between CDREM 
and GCDREM is presented in Fig. ^2 This indicates that the difference between the two 
models is almost indistinguishable even for = 3. Since the consistency becomes better as 
N grows largerl^'l^ working in GCDREM instead of CDREM is justified when is large. 

6 



A 

M 

V 

CD ^ 
O 











CDREM 


+ 






GCDREM 










^y 












1 N = 3 i 






1 a = 2 1 


-•^ 









0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 



n 



Fig. 11. The free energies obtained from the sets {nj} using CDREM and GCDREM. We find that 
CDREM and GCDREM give almost same results even for = 3. 
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